A catadioptric optical system combines refraction and reflection principles, usually via lenses (dioptrics) and curved mirrors (catoptrics). The catadioptric system includes camera imaging reflectors or refractors, enabling wide field-of-view (FOV) imaging. The catadioptric systems have been used in a wide range of applications, including panoramic imaging and visualization, wide-angle reconstruction, surveillance, and mobile robot and car navigation. The catadioptric system can be central or non-central.
Central Catadioptric System
Central catadioptric systems use a camera-mirror pair arranged to enable an effective single viewpoint, i.e., all rays of light forming an image acquired by the camera sensor intersect at one point. Examples of the central catadioptric systems include a perspective camera placed on one of the foci of a hyperbolic or elliptical mirror, and an orthographic camera placed on an axis of a parabolic mirror.
Non-Central Catadioptric System
Non-central catadioptric systems are widely used in computer vision applications. Examples of non-central catadioptric systems include a perspective camera placed outside of a spherical mirror, and configurations wherein the camera is not placed on the foci of a hyperbolic or elliptical mirror. In contrast with the central catadioptric systems, in the non-central catadioptric systems, the rays do not intersect at one point. Instead, the rays may intersect along a line, or the rays may be tangent to a circle or to a more complex shape.
In a number of applications, it is important to model non-central catadioptric systems, which, in turn, requires determining a projection of a point in a scene (PS) to a center of projection (COP) of the camera of the catadioptric system. The non-central catadioptric system does not have an effective center of projection. The COP refers to the center of projection of the physical perspective camera used in a catadioptric system.
The projection maps the three-dimensional (3D) PS to a two-dimensional (2D) pixel on an image plane of the camera of the catadioptric system.
For example, if the catadioptric system includes a reflector, such as a mirror, the projection of the PS onto an image plane of the camera requires computing a path of the ray of light from the PS to the COP via mirror reflection. Thus, the point of reflection on a surface of the mirror needs to be determined. Similarly, if the catadioptric system includes the refractor, e.g., a refractive sphere, two points of refraction need to be determined to model the projection.
Analytical solutions of projections for central catadioptric systems are known. However, there is no analytical solution of projection for general non-central catadioptric systems, when a camera is placed at an arbitrary position with respect to a reflector or a refractor.
Several conventional methods approximate non-central catadioptric systems as the central catadioptric system that enables analytical solution for projection. However, those methods lead to inaccuracies such as skewed 3D estimation.
Alternative methods use iterative non-linear optimization by initializing the point of reflection or refraction using the central approximation. However, those methods are time-consuming and inappropriate initialization leads to incorrect solutions. Yet another method uses a general linear camera representation for locally approximating a non-central catadioptric camera with an affine model that allows analytical projection, but this method also introduces approximation.
Accordingly, it is desired to provide an analytical solution of the projection for non-central catadioptric systems and to determine analytically a three-dimensional (3D) location of at least one reflection point of a ray from the PS to the COP of non-central catadioptric systems.